Papers with risk control
Controlling Risk of Retrieval-augmented Generation: A Counterfactual Prompting Framework (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on retrieval-augmented generation (RAG) rarely address the issue of predictive uncertainty, i.e., how likely it is that a RAG model’s prediction is incorrect. |
| Approach: | They propose a framework that induces RAG models to alter latent factors and analyzes the effect on their answers. |
| Outcome: | The proposed framework identifies two critical factors affecting RAG models' confidence in their answers and analyzes the effect on their answers. |
Toward Safe and Human-Aligned Game Conversational Recommendation via Multi-Agent Decomposition (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing systems for conversational recommender systems (CRS) have strong results in movies, but games present distinct challenges . MATCHA framework provides specialized agents for intent parsing, tool-augmented retrieval, multi-LLM ranking, and stronger safety. |
| Approach: | They propose a framework for conversational recommender systems that assigns specialized agents for intent parsing, tool-augmented retrieval, multi-LLM ranking and risk control. |
| Outcome: | MATCHA outperforms baselines on real user request dataset, improves Hit@5 by 20%, reduces popularity bias by 24%, and achieves 97.9% adversarial defense. |
Dynamic Simulation Framework for Disinformation Dissemination and Correction With Social Bots (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Current studies rely on simplistic user and network modeling and neglect dynamic behavior of bots. |
| Approach: | They propose a multi-agent-based framework for disinformation dissemination . it incorporates both malicious and legitimate bots and allows quantitative evaluation of correction strategies. |
| Outcome: | The proposed framework incorporates both malicious and legitimate bots and their controlled dynamic participation allows for quantitative analysis of correction strategies. |
PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preference (2025.acl-long)
Copied to clipboard
Jiaming Ji, Donghai Hong, Borong Zhang, Boyuan Chen, Josef Dai, Boren Zheng, Tianyi Alex Qiu, Jiayi Zhou, Kaile Wang, Boxun Li, Sirui Han, Yike Guo, Yaodong Yang
| Challenge: | Using large-scale annotation data, large language models can generate noise, errors and biases, leading to unexpected behaviours. |
| Approach: | They propose a dataset to promote safety alignment in large language models . they separate helpfulness and harmlessness annotations for question-answering pairs . |
| Outcome: | The proposed dataset provides 44.6k prompts and 265k question-answer pairs with safety meta-labels for 19 harm categories and three severity levels, with answers generated by Llama-family models. |